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ABSTRACT 

A considerable body of research exists on whether students 
who attend private school with a voucher benefit academically. Whether public 
schools improve in. response to the challenge of voucher programs, however, 
has been less thoroughly studied. This study provides new evidence on how 
public schools fare when faced with the challenge of school choice. School 
districts in Edgewood, Texas, and in Milwaukee, Wisconsin, that have school 
voucher programs were selected to investigate the effects of school choice on 
traditional public schools in those same districts. Student test scores in 
public schools exposed to competition by nearby charter and private schools 
were compared against control-group public schools to reduce demographic and 
economic influences on the data. Results show that public schools near 
charter and private schools respond to competition by improving educational 
services. In Texas, the Edgewood School District did as well as or. better 
than 85 percent of the Texas school districts. In Milwaukee, significant 
improvements were found in public-school test scores because of competition 
from charter and private schools. Further study is needed to bolster research 
on how school choice affects the quality of educational services provided by 
public schools. (Contains 11 references.) (RT) 
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Introduction 

A considerable body of high-quality research exists 
on whether students who are given the 
opportunity to attend private school with a 
voucher benefit academically. Random assignment 
experiments, the “gold standard” for research 
designs, have been conducted in Charlotte, 
Dayton, Milwaukee, New York, and Washington, 
D.C. to address whether school choice improves 
outcomes for students who attend private school 
with a voucher. All of those experiments show 
significant positive results, particularly for African- 
American students. 

Whether public schools improve in response to 
the challenge of voucher programs, however, 
has been less thoroughly studied. The initial 
results from a number of studies suggest that 
rather than being debilitated by a loss of 
revenue and talent to choice programs, public 
schools are energized to improve and retain 
their students. As traditional public schools are 
likely to educate the bulk of students under any 
realistic choice arrangement, more research on 
the public school response to the challenge of 
vouchers would be useful. 



This study provides new evidence on the broader 
question of how public schools fare when faced 
with the challenge of school choice. It finds that 
some public schools exposed to competition 
through school choice programs in Milwaukee and 
San Antonio significantly improved their academic 
performance. Controlling for demographic factors 
of race and income as well as for local school 
spending, public schools exposed to competition 
showed more improvement in student test scores 
than other public schools. The Edgewood school 
district in San Antonio, the one Texas district 
where public schools were exposed to a large-scale 
privately funded voucher program, did as well as 
or better than 85% of Texas school districts after 
controlling for demographics and local resources. 
In Milwaukee, private-school competition (at the 
4 th grade level) and charter-school competition (at 
the 10 th grade level) were found to cause significant 
improvements in public school test scores, again 
controlling for demographics and resources. The 
comparison between the effects of private -school 
and charter-school competition is most 
appropriate at the elementary level, the level at 
which both private and charter competition are 
widely available; at that level, only private 
competition was found to cause improvements. 




MANHATTAN INSTITUTE 







Previous Research 

CO 

O The claim most often made for school choice is 

“ that it benefits the students who participate, by 

_ enabling them to attend whichever school — public 

O or private — will provide the best education. The 

question of whether school choice benefits 
participating students has been well-studied; there 
have been five studies of school choice programs 
that have taken advantage of random assignment 
methods. Random assignment provides the most 
reliable study results, because it allows researchers 
to compare groups of students that are, for all 
practical purposes, identical in every respect except 
one: students in one group used school choice 
programs and students in the other group did not. 
All five of the random assignment studies of 
school choice found that participating students 
had significantly better test scores than non- 
participating students (see Rouse 1998; Greene, 
Peterson, and Du 1999; Greene 2001a; and Howell 
and Peterson 2002). 

Of the few studies that have been done of U.S. public schools 
exposed to school choice, none have ever found a decrease in the 
academic pef orman ce of public school students, and a few have 
found academic gains. 

However, school choice may have significant 
effects on more than just the students who 
participate. As research has accumulated showing 
that school choice benefits participating students, 
its critics have relied more and more heavily on 
the argument that school choice will hurt public 
schools and the students who remain there after 
participating students have left for private schools. 
In response to these criticisms, advocates claim 
that school choice programs indirectly benefit 
public schools and their students by forcing public 
schools to compete with private schools, providing 
a strong incentive for those public schools to 
improve. According to this argument, because 



public schools do not want to lose students (and 
the revenue students generate) to private schools, 
they can be expected to respond constructively 
to the presence of school choice programs, 
providing better educational services in order to 
reduce the number of students who choose to 
exercise their option to leave the public school 
system. The alternative claim is that the departure 
of resources and talent from public schools under 
choice programs debilitates public schools, 
harming their academic quality. 

The effect of school choice programs on public 
schools has not been studied as well as its effect 
on participating students, so less is known about 
this question. Of the few studies that have been 
done of U.S. public schools exposed to school 
choice, none have ever found a decrease in the 
academic performance of public school students, 
and a few have found academic gains. Studies of 
school choice outside the U.S. have reached less 
encouraging conclusions about its benefits for 
public education, but there are questions 
regarding whether these conclusions are 
applicable to U.S. schools. 

A previous study by Jay Greene found that failing 
Florida schools improved when threatened with 
exposure to school choice. Under Florida law, 
every school is graded each year based on its 
students’ performance on a statewide exam. If a 
school has received a grade of “F” in two of the 
past four years, students assigned to that school 
become eligible for scholarships to attend private 
schools. Thus, when a school receives its first “F’ 
grade, it knows it must provide better educational 
services or risk losing students to school choice. 
Greene found that schools that received an “F” 
in the first year of the program did remarkably 
better in the second year. For example, their 
average math score showed an improvement so 
large that it would have taken a spending increase 
of $3,484 per pupil — a 60% increase over existing 
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funds — to achieve the same improvement by 
expanding school budgets (see Greene 2001b). 
This suggests that public schools are both able 
and willing to respond constructively to the 
prospect of losing their students to a choice 
program. 

Caroline Hoxby has also studied public school 
responses to school choice programs. Studying 
charter schools in Michigan and Arizona, she 
found that public schools exposed to competition 
from charter schools (defined as public schools 
operating in areas where at least 6% of students 
were enrolled in charter schools) made 
significandy better improvements in test scores 
than public schools not exposed to charter 
competition. 

In Arizona, she found that 4 th grade math scores 
improved by 3 percentile points in public schools 
exposed to charter competition, and only by 1 
percentile point in public schools not exposed to 
competition. Hoxby also found that public schools 
exposed to choice had significandy higher gains 
in 4 th grade reading, 7 th grade math, and 7 th grade 
reading. She wrote that if schools exposed to 
competition in Phoenix continue to improve so 
much faster than schools in the Phoenix suburbs, 
the achievement gap between them would close 
in less than ten years. In Michigan, she found 
that in public schools exposed to charter 
competition, 4 th grade reading scores climbed 2.4 
scale points faster than in other public schools, 
and 4 th grade math scores climbed 2.5 scale points 
faster. She wrote that if this trend continued, 
the achievement gap between public schools in 
Detroit and public schools in the wealthy suburb 
of Grosse Pointe would close in less than twenty 
years (see Hoxby 2001). 



districts with test scores in areas served by only 
one district, such as Miami or Hawaii. In an area 
served by many districts, parents will exercise 
“school choice” on their own by finding out where 
the best schools are and moving there. Hoxby 
found that school districts faced with public- 
school competition had consistently higher test 
scores while spending less per student. In one such 
study, an increase in public school competition 
equal to one standard deviation was associated 
with a 17% decrease in per-pupil spending and a 
3 percentile point improvement in test scores (see 
Hoxby 1998). 
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This suggests that pub/ic schools are both able and willing to 
respond constructive^ to the propect of losing their students to a 
choice program. 



Most importantly, Hoxby has studied public 
school response to school choice in Milwaukee, 
one of the cities included in the present study. 
She found that schools most exposed to 
competition from private school scholarships had 
larger increases in math, science, and language test 
scores than schools less exposed to competition. 
For math scores, schools more exposed to 
competition improved by 7.1 percentile points, 
while other schools improved by 3.7 percentile 
points (see Hoxby 2001). Hoxby did not study 
public schools forced to compete with charter 
schools, as opposed to private schools, and 
includes fewer years in the period studied than 
does the present study. Also, she studied only 4 th 
grade test scores, whereas the present study also 
includes 8 th grade and 10 th grade test scores. As 
we will see, our results for 4 th grade scores are 
consistent with Hoxby’s, but we found different 
results for 8 th grade and 10 th grade scores. 



Hoxby has also measured the effects of 
competition on public schools by comparing test 
scores in areas served by many different school 



A study by Christopher Hammons of century- 
old school choice programs in Maine and Vermont 
found that such programs produced gains in 
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public-school test scores. In those states, school 
districts that do not operate their own schools 
O pay to send their students to public schools in 

~ other districts or to non-religious private schools, 

_ a practice known as “tuitioning.” Hammons 

O found that high schools located in areas with 

larger concentrations of tuitioning districts had 
better test scores than high schools with lower 
exposure to tuitioning. He found that if a town 
one mile away from a school started tuitioning, 
that school could expect a 3.4 point increase in 
test scores, a gain of 12% over existing scores 
(see Hammons 2001). 

Studies examining public school response to school choice in other 
countries have found more mixed results. However, there are 
serious questions about whether the programs covered by these 
studies are ana/ogous to school choice programs in the U.S. 



Studies examining public school response to 
school choice in other countries have found more 
mixed results. However, there are serious questions 
about whether the programs covered by these 
studies are analogous to school choice programs 
in the U.S., both because of differing national 
conditions and because the programs themselves 
are different. A school choice program in New 
Zealand, studied by Helen Ladd and Edward Fiske, 
did not create many new schools and provided 
choice only within the traditional public school 
system; furthermore, teachers’ contracts and the 
academic curriculum were both controlled by the 
national government and kept uniform across all 
schools, allowing for little meaningful choice even 
within the public school system (see Ladd and 
Fiske 2000). A private-school scholarship program 
in Chile, studied by Patrick McEwan and Martin 
Carnoy, provided so little money for the 
scholarships that in practice students had only 
limited opportunities to use them; what’s more, 
the study failed to control adequately for student 
demographics and background, making it 



impossible to separate the effects of school choice 
from the effects of poverty, family status, and 
other factors (see McEwan and Carnoy 1999). 

Frederick Hess has conducted case studies 
suggesting that while U.S. public schools do 
respond to school choice, institutional and cultural 
factors limit the significance of these responses. 
Hess did not look at test scores, focusing instead 
on whether public schools undertake institutional 
reforms, and most of his explanations for public 
school resistance to change are theoretical. In 
addition, he studied cases in which school choice 
had not been in effect for long periods of time. 
However, his work does shed light on potential 
barriers to change in public schools (see Hess, 2002). 

Method 

Edgewood is a small school district in San 
Antonio, Texas, serving a low-income, Hispanic 
section of the city. Since 1998 a private 
organization has offered a scholarship to every 
student in the Edgewood district with which 
students could attend a private school or a public 
school in another district. This scholarship 
program effectively creates a system of school 
choice in Edgewood in which all schools in the 
district are exposed to private-school competition. 

In order to compare schools that were exposed 
to competition (“treated” schools) with schools 
not exposed to competition (“control” schools), 
we had to compare the performance of the 
Edgewood school district with that of other 
school districts in Texas. This required us to 
develop a model that would control for the 
demographic characteristics of each district’s 
population, as well as the level of resources 
available to public schools in each district. 

To accomplish this, our model calculates an 
“expected gain” for each district in test scores on 



the statewide TAAS test between 1998 and 2001, 
the period covered in the study. This expected gain 
is calculated based on the demographic 
characteristics of the district population (including 
race and poverty) and local resources (including 
the district’s per-pupil spending and teacher- 
student ratio). For each of these factors we took 
into account both the district’s characteristics at the 
beginning of the study period and the change in 
those characteristics over the study period. The 
expected gain figure for each district allows us to 
control for the effects of population demographics 
and local resources. By comparing a district’s actual 
test score gain (or loss) with its expected gain, we 
can isolate changes in test scores that are not 
attributable to changes in population or resources. 

To the extent that Edgewood outperforms its 
expected gain, some factor other than changes in 
population and resources has pushed up its test 
scores. Likewise, to the extent that Edgewood 
underperforms its expected gain, some other factor 
has driven down test scores. And by ranking all 
districts in Texas according to how well they 
outperformed (or how badly they underperformed) 
their expected gains, we can see whether Edgewood 
is outperforming its expected gain significantly 
better (or underperforming it significantly worse) 
than most Texas districts. To the extent that 
Edgewood performs significantly stronger or 
weaker than most Texas districts, this would be 
consistent with the hypothesis that school choice 
has had a significant effect on the quality of public 
education. Of course other factors, including mere 
chance, may play a role in determining Edge wood’s 
statewide rank. However, the further Edgewood 
ranks towards the top or bottom of the scale, the 
more likely it is that school choice had some 
significant effect. 

Milwaukee has the largest school voucher 
program, and, with the exception of “tuitioning” 
programs in Maine and Vermont, is the longest 



running voucher program. In Milwaukee, unlike 
in Edgewood, not every school was equally 
exposed to competition from choice programs. 
This allowed us to adopt a different (and better) 
research design in which we were able to examine 
whether schools that were more exposed to 
competition from choice programs experienced 
greater academic improvement. We measured both 
the extent to which each school was exposed to 
private-school competition and the extent to 
which each school was exposed to charter-school 
competition. 
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To the extent that Edgewood performs significantly stronger or 
weaker than most Texas districts, this wou/d be consistent with 
the hypothesis that schooi choice has had a significant effect on the 
quality of public education. 



To measure the extent to which each school was 
exposed to charter-school choice, we used an index 
of the distance between the school and the three 
nearest charter schools. The closer a public school 
is to charter schools, the more it is exposed to 
charter-school competition. 1 Distance to charter 
schools is the relevant measurement of exposure 
to charter- school competition because all charter 
schools in Milwaukee are free and all students are 
eligible to attend them. The only variation in access 
to charter schools is logistical: how hard is it for 
students to get to the charter school? If a public 
school is located close to charter schools, less extra 
effort is required for students to attend those 
charter schools rather than the public school. The 
easier it is for students to get to charter schools, 
the more likely the public schools are to face 
competition from those schools. 2 



We faced a challenge in measuring exposure to 
private-school choice, because all students 
participating in the federal free and reduced-price 
lunch program were qualified to receive private 
school scholarships. We had to use participation 
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in the federal lunch program both to measure 
exposure to private-school choice and to control 
for the effects of poverty in the school population. 
To solve this problem, for each school we used 
the percentage of the school’s population that 
participated in the federal lunch program in the 
1995-96 school year as a measurement of the 
extent to which that school was exposed to private- 
school choice, and the percentage of the school’s 
population that participated in the federal lunch 
program in the 2000-01 school year as a 
measurement of poverty in the school population. 
We did so because 1995-96 was one year before 
the city’s private- school choice program expanded 
to full scale. Thus the federal lunch program 
participation figure for 1995-96 represents the 
number of students in each school who were 
about to become eligible for private-school choice, 
while the 2000-01 figure represents the number 
of impoverished students remaining in each 
school after school choice became widely available. 



This is consistent with the hypothesis that public schoo/s are 
responding to competition from schooi choice by improving 
educationai services. 



Our study measured whether changes in each 
school’s test scores for 4 th grade, 8 th grade, and 
10 th grade students on the Wisconsin Knowledge 
and Concepts Examinations between the 1996- 
97 and 2000-01 school years were related to our 
measurements of exposure to private-school and 
charter-school competition. We used the 1996-97 
test scores, rather than earlier test scores, because 
changes in the test rendered earlier scores non- 
comparable with 2000-01 scores. In the model, 
we controlled for race and poverty in each school’s 
student population, as well as for projected per- 
pupil spending for the 2002-03 school year. A 
statistically significant relationship between 
exposure to competition and changes in test scores 
over the study period, controlling for other factors, 
indicates that competition had a significant effect. 



Results 

Our findings for Edgewood are summarized in 
Table 1. We found that when Texas school districts 
were ranked according to the difference between 
actual gains and expected gains over the study 
period, the Edgewood school district ranked at 
the 85 th percentile. This means that Edgewood’s 
performance relative to its expected gain was equal 
or superior to that of 85% of all Texas school 
districts. Among Hispanic students, who made up 
97% of Edgewood’s population, Edgewood 
ranked at the 73 rd percentile statewide. Among 
lower-income students (those in the federal lunch 
program), who made up 93% of Edgewood’s 
population, Edgewood ranked at the 75 th 
percentile statewide. 



Table 1 

Statewide Rank of Edgewood School District 
(Actual Gain Relative to Expected Gain) 




Edgewood 


Population 


Statewide Rank 


All Students 


85th Percentile 


Hispanic Students 


73rd Percentile 


Impoverished Students 


75th Percentile 



This indicates that, after the effects of population 
demographics and local resources were isolated 
and removed, Edgewood performed well above 
the average Texas school district among all 
students, Hispanic students, and low-income 
students. This is consistent with the hypothesis 
that public schools are responding to competition 
from school choice by improving educational 
services. Of course, other factors may be at work, 
including random chance. However, given 
Edgewood’s unusually strong performance, the 
data suggest that school choice probably made an 
important difference in student outcomes. 

Our results for Milwaukee are summarized in 
Table 2. In 4 th grade, we found a statistically 
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significant relationship between private-school 
competition and public school test scores, but not 
between charter-school competition and public 
school test scores. 3 The effects of private-school 
competition were such that if a public school had 
only 50% of its students eligible for private- school 
scholarships in 1995-96, and was in every other 
way average, it could expect its average test score 
to decrease by just over 5 percentile points over 
the study period. On the other hand, if the same 
public school had 100% of its students eligible 
for private -school scholarships in 1995-96, it could 
expect its average test score to increase by just 
over 10 percentile points — a 15-point difference. 
(On average, the actual level of exposure to 
private-school competition was about 77% for 
elementary schools.) 4 



However, our results for middle and high school 
tests were not identical to our results for 4 th grade 
tests. In 8 th grade, we found no statistically 
significant relationship between exposure to either 
private- school or charter-school competition and 
test scores. And in 10 th grade, our findings for 4 th 
grade were actually reversed: we found a significant 
positive relationship between charter-school 
competition and public school test scores, but no 
significant relationship between private-school 
competition and test scores. The effects of 
charter-school competition were such that if a 



school faced competition from only one charter 
school located 5 km away, and was in every other 
way average, it could expect its average test score 
to increase by only about 3.5 points between 1996- 
97 and 2000-01. But if we were to pick up that 
competing charter school and move it 4 km closer, 
such that it was only 1 km away, the public school 
could expect its average test score to increase by 
about 9 points over the study period. 5 

In interpreting these results, two important facts 
need to be kept in mind. The first is that there is a 
much greater supply of private elementary schools 
than of private middle and high schools. This is 
due primarily to the significandy greater per-pupil 
spending needed to operate a middle or high school. 
In light of this fact, the discrepancy between our 
results for 4 th grade, 8 th 
grade, and 10 th grade is less 
mystifying; private- school 
competition seems to apply 
more competitive pressure 
on public schools at the 
elementary level, where 
there are more private 
elementary schools for 
parents to choose from, 
and less competitive 
pressure at the middle and 
high school level, where 
charter schools take up 
some of the slack caused by the smaller number 
of private options available. At the elementary level, 
the level at which both private and charter 
competition are available, private schools rather 
than charter schools seem to drive public school 
improvements. 

The other fact to be borne in mind is that our 
results for 8 th grade and 10 th grade tests are 
hindered by the low number of public middle and 
high schools in Milwaukee. While we had data 
from 115 public elementary schools in the city, 



Table 2 

Average Test Score Change In Public 
Schools Associated With Competition 




4th Grade 


Expected Test 
Score Change 


50% of students eligible for private school scholarships 


-5 points 


100% of students eligible for private school scholarships 


+ 10 points 


10th Grade 


Charter school 5 km away 


+ 3.5 points 


Charter school 1 km away 


+9 points 
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we had data from only 33 middle schools and 20 
high schools. This smaller number of schools 
reduces the chance that we will discover any 
relationship that may exist between exposure to 
competition and test scores. 

Conclusion 

The data in this study seem to indicate that public 
schools respond to competition from school 
choice programs by improving educational 
services. In Texas, the one school district where 
public schools were exposed to private-school 
competition did as well as or better than 85% of 
Texas school districts, after controlling for 
population demographics and local resources. In 
Milwaukee, private-school competition (at the 4 th 
grade level) and char ter- school competition (at the 



10 th grade level) were found to cause significant 
improvements in public school test scores, again 
controlling for demographics and resources. At 
the elementary level, for which both private and 
charter competition are widely available, only 
private competition was found to cause 
improvements. 

There is, of course, much that these data do not 
tell us. Further study of the subject is badly needed, 
especially since much less research has been done 
on this question than on the question of school 
choice’s benefits for students who directly 
participate in choice programs. Given that whether 
school choice helps or hurts public schools is a 
question of large and growing political 
importance, discovering more about this 
phenomenon should be a high research priority. 
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Endnotes 

1. We measured exposure to charter-school competition by taking the sum of 1/X + 1/Y + 1/Z, where 
X, Y, and Z are the distances in kilometers between the school and each of the three closest charter 
schools. Thus, the further away the three closest charter schools were, the lower the figure for charter 
school competition. 

2. This research design uses the distance from a public school to a charter school as a proxy for the 
distance between that charter school and the homes of the students who attend that public school. For 
this to be true, it must be the case that most Milwaukee public school students attend schools close to 
where they live. Aquine Jackson, director of the Milwaukee Public Schools Office of Neighborhood 
Schools indicates that this is the case (phone interview, 1:15 pm, Sept. 30, 2002). According to the 
Office of Neighborhood Schools, only 63% of Milwaukee public school students are bused to school. 
Elementary school students are bused if they live one mile from school, middle school students are 
bused if they live two miles from school, and high school students are bused if they live outside the 
school’s attendance area. Since students living as close as one mile away from school can be bused, the 
63% figure includes many students who live close to their schools; clearly most Milwaukee public 
school students go to school close to where they live. 

3. Following standard practice, for all regression analyses we considered a result statistically significant if 
p was less than 0.05. 

4. For our 4th grade results, the standardized coefficient for exposure to private- school choice was 0.49. 
This means that an increase of one standard deviation in exposure to private-school choice can be 
expected to produce an increase of about half a standard deviation in a school’s average test score gain. 

5. For our 10th grade results, the standardized coefficient for exposure to charter-school choice was 
0.59. This means that a difference of one standard deviation in exposure to charter-school choice can 
be expected to produce a difference of about six-tenths of a standard deviation in a school’s average 
test score gain. 
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